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ABSTRACT 

The influence of the Three Week Residential Seminar 
in PPB offered by the civil Service Commission on the job 
performance, attitudes, and career development of participants who 
attended the first 15 seminars was studied. The seminar program was 
developed to retrain experienced analysts (financial, budget, and 
management) and PPB output user managers for immediate assignment in 
and near newly created PPBS units. Questionnaires were sent to 653 
former participants, representing 90 percent of the 722 individuals 
who attended the 15 sessions from February 1966 through June 1967; 
the mean response was 79 percent with a range of 66 to 93 percent. 

The characteristics of the participants were as follows: average 
level of education was four and one- half years of college; average 
grade level was GS-14, with a range from 9 to 18; average age was 44 
years with a range of 24 to 67 years; the average years of work 
experience in the Federal government was 18 and the range was 2 to 40 
years; and 25* had attended at least one additional course related to 
PPBS in the past two years. The results of the study showed that (1) 
a majority of the participants were directly engaged in some phase of 
PPB operation, (2) the seminars provided understanding of PPB 
concepts, (3) the techniques presented in the seminar had been put to 
use, (4) the seminar had stimulated self- development efforts related 
to PPB, and (5) 35% were able to describe specific job outputs 
traceable to seminar teaching. Statistical analysis of the results is 
presented in tables. Appendixes present the seminar schedule, 
questionnaire, and guest. (DB) 
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I. INTRODUCTION 



Development of the PPBS Seminar 



With the Presidential memo to the Cabinet members and Agency 
heads in August 1965, the Planning, Programming, Budgeting System, 
which had been adopted in the Department of Defense, became an item 
of primary concern to the non-military Departments and Agencies in 
the executive branch. In the memo, two major objectives for PPBS 

were cited: 

(1) To identify national goals with precision. 

This involved substantially more than the 
kind of general approach that had been 
sufficient in the past. The goals of the 
Federal community would have to be stated 
in terms against which progress could be 
measured. 

(2) To attain specifically identified national 
goals at a minimum level of resource expen- 
diture. This was a departure from the 
familiar approach of taking a given amount 
of resources and trying to accomplish with 
these resources as much as possible. 



The Presidential memo was followed by Bureau of the Budget 
Circular 66-3, published in October 1965, directing 22 Departments 
and Agencies to move ahead immediately and develop a PPB System 
that would produce program memoranda by May 1, 1966. 
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> addition, seventeen other Agencies were encouraged to apply 

i?P3 principles and procedures for the development and review of 
programs to the extent practicable. 

In the Fall of 1965, no interagency training in PPB existed. 
The only training in Government had been started at the Naval 
Postgraduate School, Monterey, California in August 1965. In 
October 1965, this program was made available to 60 personnel 
from the non-military organizations mentioned in BOB Bulletin 66-3. 
One of the participants was Chester Wright, an employee of the 
Service Commission’s Office of Career Development, and the 
only training specialist in attendance. 

In January 1966, William A. Medina, also employed in the 
Office of Career Development, attended the course, and in February 
1966 the Civil Service Commission offered the first non-military 
interagency Three Week Residential Seminar in PPB. It was held at 
the University of Maryland, using faculty from the College of 
Business and Public Administration, in combination with consultants 
and Government specialists. The Bureau of the Budget provided close 
guidance throughout the development of the course and key personnel 
have participated as guest lecturers. 

Between February 1966 and May 1968 , 1095 persons fro® 28 
Departments and Agencies had attended the course conducted under the 
direction of the Office of Career Development later known as the 
Bureau of Training. 

ERJC 



6 



-3- 



This seminar is part of the wider plan for PPBS training, 
developed by the Commission in cooperation with the Bureau of the 
Budget, that also includes orientation for Government executives, 
long-term graduate education for analysts, and short, specialized 
interagency courses* 

Most of the seminars included in this follow-up study were 
conducted at the University of Maryland, and the University of 
Virginia. Some were held at Harvard University, and at commercial 
conference sites in the Washington, D. C. area. 

It must be emphasized that the training program that is 
being evaluated is not static. Many improvements were made as the 
first 15 seminars were being planned and conducted and this process 
has continued to the present time. In the field, the first shorter, 
but parallel, seminar in PPB was conducted by the San Francisco 
Civil Service Region in early 1968. 

Purpose and Objectives of the Seminar 

The seminar program was developed to retrain experienced 
analysts (financial analysts, budget analysts, management analysts) 
and PPB output user managers for immediate assignment in and near 
newly created PPBS units. 
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Th e objectives of the Planning, Programming f Budgeting 
Seminar are: 

(1) To provide the student with a grasp of 
the underlying economic base of program 
budgeting. 

(2) To provide a working knowledge of the 
structure and functioning of the Planning, 
Programming and Budgeting System as set 
forth in the Bureau of the Budget Bulletins 
with particular emphasis on the long-range 
planning aspects of that System. 

(3) To introduce the student to quantitative 
approaches to management planning and 
control, and improve his ability to communi- 
cate intelligently with quantitative 
analysts. 



The essential distinction of PPBS is that it is truly inter- 
disciplinary. An appreciation of the system by the participants 
demands that they have an awareness of the disciplines and how they 
have been synthesized. The component disciplines are presented in 
the course and their relevance is demonstrated through perceived 
interrelationships that grow naturally out of the course content and 
use of case materials. 



Summary of Seminar Content 

The three segments of the course are arranged with a logical 
and progressive interrelationship. The first segment provides the 
student with an understanding of the underlying philosophy of PPBS. 
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He acquires an understanding of the functioning of the system 
and is introduced to the fundamental economic concepts involved. 

He is also introduced to the concept of long-term financial and 
program planning, to some of the requirements for successful 
long-range planning, and to some of the benefits of this manage- 
ment process. 

The second segment provides the student with a fairly 
detailed look at some of the more significant concepts of economic 
analysis, along with an introduction to the computational processes 
Involved in contemporary economic analytical techniques. 

The third segment continues the examination of quantitative 
problem-solving approaches through the primary medium of case 
studies developed specifically to provide insight into analytic 
techniques associated with PPBS and with management uses of the 
products of analysis. 

For additional information about course content, see the 
seminar program in Appendix A. 
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Development of Instructional Materials 
for the Seminar 

Early In the first year of the course, it was recognized 
that a comprehensive set of case studies would be necessaiy to 
successfully accomplish the program objectives. Consequently , 
arrangements were made for development of cases by Harvard Univer- 
sity. Funds were supplied by the Civil Service Commission and the 
Bureau of the Budget. 

The cases were completed in mid- 1966 and have been exten- 
sively used in the seminar since then. In addition, the materials 
have been made available to the public through the Harvard Univer- 
sity Case Clearing House. During the past year cases have been 
purchased by state, local, and foreign governments. Work with 
some of the cases has also been included in the required curriculum 
of the Harvard Business School. 

The most recent phase in the story of instructional materials 
is a $25,000 project funded by the Ford Foundation for the develop- 
ment of cases that relate directly to the application of PPB in 
state and local government. 



ERIC 
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II. PURPOSE OF STUDY 

The purpose of this study is to evaluate the influence 
of the three week seminar on the job performance, attitudes, and 
career development of participants who attended the first 15 
seminars. 

We obtained reactions from employees of many Agencies and 
Departments who had been out of the course for sufficient time to 
apply what they may have learned either where P?B had been in- 
stalled, or where their duties gave opportunity for applying PPB 

concepts in advance of a total system. 

Specifically* we wanted answers to the following questions: 

1. Are former participants now working 
directly in PPB? 

2. Has the total course or elements of it* 
been of benefit on the job? 

3. Given time for reflection and comparison, 
do participants think the course gave 

' them an understanding of how various 

economic, analytic, and administrative 
concepts are integrated to form the PPB 
system? 
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Do participants feel that the 
course was instrumental in changing 
their way of thinking about govern- 
ment programs, and if so, along what 
dimensions? 

Have quantitative and analytic tech- 
niques presented in the course been 
of specific value to participants? 

Do participants attribute specific 
job output ^ to the training? 

Have the participants pursued addi- 
tional study in subjects related to 
the course, and if so, what subjects? 
Do participants desire additional 
follow-on training in PPBS, and if so, 
what kind? 
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9. What is the relative influence of 
age, grade, educational level, 
length of government experience, 
program area, and type of work on 
participants* response to the 
course? 

10. What specific recommendations do 
the participants have about changes 
in the training program structure, 
content, and subject matter emphasis? 

Answers to the foregoing questions would be critical for 
deciding continuance of the course, for improving the structure 
of the course, and for sharpening the guidance to Agencies selecting 
participants. The answers would also be vitally important to the 
development of follow-on training. 
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ll I. SYNOPSIS OF RESULTS 

A substantial majority of those who attended the Three 
Week Seminar in PPBS before July 1967 were found to be directly 
engaged in some phase of the PPB operation in their Agencies. 

The seminar generally succeeded in giving the partici- 
pants an understanding of what theoretical concepts underlie 
planning -programming -budgeting, and how these concepts have been 
synthesized to produce a decision-making system. 

There has been widespread attitudinal change by the former 
participants concerning outputs, costs, alternatives, and objec- 
t ives . 

The quantitative and systems analysis techniques presented 
in the course have been put to use. 

There has been progressively greater satisfaction with the 
balance and emphasis of course content as tima passed and the 
structure was refined. 

The seminar has been successful in stimulating self -develop- 
mental efforts in areas related to PPB. 

A substantial number (35 percent) were able to describe 
specific job outputs traceable to seminar teaching. About half of 
the group offered specific srggestions about the kinds of addi- 
tional training would be useful to them. A large majority of those 

who have completed the seminar have recommended it to and for their 
co-workers. 
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More emphasis on the practical and less on theory was 
the prevalent voluntary comment. 

Information at hand at the start of this study, and data 
collected through the questionnaire, indicated that there has 
been a very wide variance in the kinds of people who have 
attended the PPBS Seminar. Their age, education, and years of 
experience all encompass nearly every point on the continuum 
found in the career Federal service. Men who had no college and 
those with Ph.D's, plus, were classmates. Individuals under 25 
years old and some over 65 were trained. Experience ranged from 
2 years to 40. There was an equally impressive variety of occu- 
pations represented in the fifteen classes under study. 

The existence of such wide dispersions presented an oppor- 
tunity to explore some assumptions about the impact of training 
on older people vis-a-vis younger; the importance of selecting 
college educated people for conceptually oriented training; the 
state of readiness for new ideas of relatively new employees 
compared with those with considerable experience; and other similar 
assumptions. The opportunity was exploited by selecting the 
questionnaires returned by those in the extremes and comparing the 
replies. Some statistically significant differences were detected, 
but the number of differences was surprisingly small. 

Additional comparisons on the basis of major program area 
and occupation also produced few distinctions that could be attri- 
buted to real difference rather than chance. 
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Ivf. CHARACTERISTICS OF THE PARTICIPANTS 



Most of those who attended the Seminar were college 
educated. The average level of education was four and one-half 
years of college with 37 percent having completed five or more 
years of college. Nine percent had no college. 

The average General Schedule grade was GS-14 and the 
range was OS-9 to GS-18. Thirty-one percent have been promoted 
to the next higher grade since completing the Seminar in an 
average elapsed time of 13 months. This compares favorably with 
the 13.5 percent promotion rate for the segment of the entire 
Federal work force in the same grade range during the 12 months 
of FY 1967. 

Forty-four years was the average age of the participants 
with a range of 24 to 67 years. The median age was 45.5 years 
and the mode was 46 years. The average participant had 18 years 

work experience in the Federal Government aid the range was 2 to 
40 years. 

The group has been active in other training as well. 
Twenty-five percent have attended at least one additional Govern- 
ment-sponsored course related to PPBS within the past two years. 
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Civil Service Commission courses frequently reported were 
Mathematics for Managers, Cost /Benefit Workshop, and Executive 
Orientation in PPBS. 

In the following sections, other participant character- 
istics, such as occupational specialty and kind of academic 
preparation, will be summarized and analyzed in relation to 
training results. 
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V. INTERPRETATION OF RESULTS 



This section gives tentative answers to the ten questions 
posed in the section entitled, "Purpose of Study." These answers 
are the product of careful analysis of the data contained in Part 
B, "Presentation of Results and Analysis." 

The Present As signmen t of Past Participants 

Nearly all former participants who returned the question- 
naire were engaged in PPB, either directly or indirectly, at the 

ic 

time they responded. Sixty-two percent reported direct assignment 
in PP3 functions and an additional 37 percent reported indirect in- 
volvement. There were few distinctions between the responses of 
these two groups about the value and utility of the course and 
those distinctions that did occur involved parts of the course that 
dealt with specific tasks related to the PPB function. 



* 

Those who did not respond did not appear to be following different 
occupational patterns at the time they attended the course than those 
who did respond. 
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The Gen eral _Xm£a£t 

The evidence indicates that the program has been 
reasonably successful in achieving its stated objectives. The 
great majority of former participants reported that they do 
understand and appreciate the interrelation of the varied 
concepts, techniques, and skills which form the PPB system. 

Over half said they left the course with a "good understanding" 
of this interrelationship, and this response was nearly uniform 
throughout the subclassifications of respondents. 

Seventy-five percent of the former participants who 
returned questionnaires felt that their way of thinking about 
Government programs in terms of alternatives had been changed. In 
addition, between 60 and 65 percent indicated a new perspective 
concerning the other areas provided for choice: outputs, costs, 

and objectives. 

The proportionate distribution of responses to this question 
was essentially the same regardless of the criteria except in the 
case of those working in International programs where the frequency 
of responses concerning outputs, costs, and objectives was somewhat 
lower than from the other program groups. 



•k 

For additional information about the criteria and analytical 
approach, please see Method of Study in Part B. 
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Over 50 percent have found the quantitative and systems 
analysis techniques presented In the course helpful In their 

«ork in at least two situations, problem analysis and identifying 
and considering alternatives. 

The course has stimulated the interest of the participants 
to the extent that 58 percent had continued to study in subjects 
related to PPB through outside courses, personal readings, etc. 
Finally, 81 percent of those who attended the course and returned 

the questionnaire have recommended that others in their Agency 
take the training. 

— £ffect of s pecif icCour se Elements 

Economic Concepts and Analytic Techniques were the two 

course elements most frequently cited as being of professional 

benefit. These elements were chosen by 49 and 53 percent of the 

respondents respectively. The element chose least frequently (by 

22 percent) was Program Memoranda and Program Financial Plan 

Preparation. The other elements in order of their selection were 

Program Structuring, Quantitative Techniques, and Information 

System Concepts. This pattern of response prevailed in most 

instances when the selections were analyzed by age, grade, occupa- 
tion, etc. 
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Those in higher grades (GS-16 and above) tended to favor 
Economic Concepts slightly more. Over 50 percent of the lower 
grade group (GS-12 and below) did not cite either. Comparison 
of responses from those working directly in PPB and those not 
working directly in PPB indicated that both groups got greatest 
benefit from Analytic Techniques and Economic Concepts, in that 
order. Those working directly in PPB, however, selected Economic 
Concepts to a significantly greater extent than those not in PPB. 
All of the elements provided for choice were selected more 
frequently by those working in PPB, except Information System 

Concepts. 

We found exceptions to the order of selection (Analytic 
Techniques, first; Economic Concepts, second; and Program Memo- 
randa and Program Financial Plan Preparation, last) in the replies 
whan we grouped them by educational background. In this case, 
everyone but those with no degree thought the Economic Concepts 
were most beneficial. The Economic Concepts were of significantly 
greater benefit to those with Scientific and Technical education 
background than to any other group, except those with Liberal Arts 

preparation. 

Those respondents who attended the first three sessions of 
the seminar found Economic Concepts to be of greatest professional 
benefit. But, those who attended the last three sessions rated 
Analytic Techniques over Economic Concepts. 
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TIk^ latter group also selected Program Structuring over Economic 
Concepts in spite of the fact that a significantly greater number 
of them were not worxing directly in PP3. 

Economic Concepts also proved to be of greatest professional 
benefit to those working in Natural Resources and Scientific and 
Technical program areas. Those working in International programs, 
conversely, selected Economic Concepts to a significantly lesser 
extent. Their second choice after Analytic Techniques was Program 
Structuring, but not significantly so. 

Former participants from Technical occupations rated Economic 
Concepts as being most beneficial. They also selected Program 
Memoranda and Program Financial Plan Preparation with a significantly 
higher frequency than did the other occupation groups. 

These observations about the utility of the course content 
present a curious contradiction to the comment by 24 percent of the 
respondents who said that theoretical aspects of PPB should be de- 
emphasized and that the course content should be more specifically 
oriented to the participating Agencies. 

An understanding of PPB depends on an understanding of how 
contributions from several disciplines have been synthesized to pro- 
duce a powerful decision making tool. Over one half (51 percent) of 
the former participants who returned the questionnaire indicated that 
they left the course with a "good understanding" of the integration 
of the various concepts. This opinion was given in the light of 
subsequent observation and comparison. 
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Our analysis of the responses according to all the 
criteria produced parallel response patterns. Both the signifi" 
cant ly younger and significantly older groups indicated "good 
aiderstanding" with a frequency higher than the average of all 
responses. Of the two age groups, the frequency of "good under- 
standing" by those significantly younger was only slightly higher. 

Respondents in grades GS-12 and under and GS-16 and above 
indicated a "good understanding" less frequently than the average, 
w ith less than average frequency of responses indicating they 
"didn’t really understand," making "fair understanding" the preva- 
lent choice. Fifty percent of the junior group and 55 percent of 
the senior group indicated "fair understanding." 

In the analysis by education, those with no college degree 
reported the lowest frequency of "good understanding." The bache- 
lor of science group was also below average. The other education 
groups reported "good understanding" with a frequency above 
average. The differences among all education groups were not 
significant. 

The experience factor provided some distinction although it 
was not significant. While those with five years or less and 
thirty years or more experienced both expressed "good understanding 
with above average frequency, the junior group expressed a notice- 
ably higher frequency of good understanding than did the senior 

group. 
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Of those who attended the first three and last three 
seminars subject to this survey, a higher than average number 
felt they left the course with a "good understanding." There 
was only a slight difference between the frequency of responses 
of these two groups, the latter being higher. 

Analysis of responses to this question by occupational 
grouping also failed to prodjce a significant distinction. The 
frequency of the "good understanding" response, however, was 
noticeably higher from the Financial and Staff Support groups 
while the frequency of this response from Line Management and 
Technical occupation was noticeably below average. 

Tho Applicat ion o f Quantitative and Systems 
Analysis Techniques 

We found situations calling for identification and selection 
of alternatives to be the most prevalent occasions when the quanti- 
tative and systems analysis techniques presented in the course were 
helpful. Fifty-eight percent made this selection. One-half of the 
respondents also chose problem analysis. The other choices in the 
order of their selection were: communicating with others, problem 

structuring, and problem solving. The frequency was 43 percent 
down to 28 percent. 

The relative distribution of responses among the choices 
provided was similar under each of the criteria provided for analysis. 
There were, however, some significant distinctions among the groups 
under some of the criteria. 

ERIC 
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Those who attended the last three sessions selected problem 
structuring with a significantly lower frequency than did those 
who attended the first three sessions. Those with thirty or more 
years service selected it significantly less frequently than did 
those with five years or less service. Those with no college 
degree and those with a degree in business administration or public 
administration selected problem structuring with a significantly 
lower frequency than the other education groups. 

Problem solving and identification and selection of alter- 
natives also proved to be distinguishing situations under two 
criteria, education and occupation. Those with no degree selected 
problem solving with significantly less frequency than did those 
with degrees in science or business or public administration. The 
M no degree" group's selection of identifying and considering alter- 
natives was equally different from that of the business and public 
administration group. Under occupation, the Financial group was 
significantly lower in its frequency of selection of problem analysis 
than the Staff Support group, and significantly lower in selection 
of identifying and considering alternatives than any of the other 
groups. 

Specific J ob Ou tputs 
Attribu ted to th e Training 

About 35 percent of those who returned the questionnaire felt 
that there had been at least on instance where there had been tan- 
gible results that could be attributed to the training. 

O 
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The outputs fell within three basic categories: (1) analysis; 

(2) preparation of plans and memoranda and related documents; and 

(3) comwu tieating and training. These categories were defined 
by key words and phrases used by respondents in describing their 
outputs. 

Analysis of the responses by experience, program area, occu- 
pation, and the dates of course attendance did not produce an> 
meaningful distinctions. The only significant difference occurred 
when the answers of those working directly in PPB and those not 
working directly in PPB were compared. Forty percent of those in 
PPB cited outputs, whereas 27 percent of those not in PPB felt they 
had produced something that they could directly relate to the 
training. 

Ad ditional Study that can _be 
Related to the Course 

Over one-half (58 percent) of those who responded reported 
that they had attempted to build upon the knowledge and understanding 
they had gained from the course through additional study in a formal 
classroom situation, by following an organized reading plan, or by 
devoting additional attention to relevant books, periodicals, and 
journals. The subject matter areas, in the order of frequency of 
mention, were analytic techniques, PPB as an Integrated concept, 
economics, numerical science, and management and administration. 

Analysis of the responses according to the various criteria 
gave results essentially similar to the overall average of 58 percent 
seeking additional study. 

o 
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Exceptions occurred in the case of the significantly younger 
participants where 70 percent had pursued further study as well 
as with those having five years or less experience (63 percent). 

Within the education criteria those with no college degree 
and those with degrees in business administration and public 
administration reported further study significantly less fre- 
quently than those with other kinds of education who were above 
the 58 percent average. Those not working directly in PPB also 
reported further study to a significantly lesser extent than did 
their fellow former participants who had been working in PP3. In 
spite of the difference, those not in PPB reported an impressively 
high 48 percent rate of participation in additional study which 
had been stimulated by the course. 

The Kind s of .Follow -on Tra ining 
Ne ed ed by Former P artic ipants 

Among the 51 percent who did request additional training, the 
most frequently mentioned approach was a shorter program that would 
include more detailed treatment of certain portions of the curricu- 
lum of the three-week seminar. Among the alternatives available 
within this framework, the most popular was a short course specifically 
designed to respond to the needs of an Agency or of Agencies all in- 
volved in the same program area. 
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(An experiment with one-week, program oriented seminars -during 
the Spring of 1968 was not successful due to lack of Agency 
response) . 

Other subjects for follow-up training included analytic 
techniques, program memoranda and financial plan preparation, and 
numerical science. 

Open -en d Comments About 
th e Three Week P?B Seminar 

No further comment or suggestions’* was the most frequent 
reply (37 percent) to the request for other observations or sug- 
gestion regarding the seminar. Specific comments about course 
content was the next largest category of responses (24 percent) 
followed by generally complimentary remarks (22 percent) and comments 
about course administration (11 percent). 

The aspect of course content of primary concern was that of 
theoretical versus practical emphasis, and the consensus was the 
desirability of more practical and less theoretical. The second 
most frequent comment about course content called for subject matter 
treatment more closely oriented to the situations of the partici- 
pating Agencies. 

Time was the most often mentioned feature of the course 
administration. Opinion was divided, however, on whether more or 
less time was required. 
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VI. DECISIONS BASED ON RESULTS 

The usual purpose of a detailed examination of past 
behavior is to improve related behavior in the future. In this 
instance, the painstaking evaluation of the 3-week PPB seminar 
produced information which hai^ already influenced management 
decisions regarding uhe structure and content of future PPB 
training. We have made changes in the 3-week seminar, including 
shortening it to 2 weeks, and we have added certain new technical 
courses to the PP3 curriculum. 

Even before collecting and analyzing information through 
the course evaluation, the training staff had perceived indications 
of the need for expanded technical training, and adjustment in the 
length of the 3-week course. The preliminary results of the study 
contained confirming evidence of these needs, and before the con- 
clusion of the project steps were taken to put some of the more 
obvious improvements into effect. 

First, we had created a cost/benefit workshop, and then a 
management information technology course. We are now In the pro' 
of re-designing the cost/benefit workshop to make it more responsive 
to past participants' expressed needs and to inclade some relevant 
case material . 
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Analysis of the evaluation results also sharpened the 
evidence that additional technical training is needed. We have 
already made or are now planning the following additions to the 
curriculum: 

. Scientific Cost Analysis 
. Decision Model Building Workshop 
. Work Force Estimating 
. Orientation to System Analysis 
. Systems Analysis Techniques 

Other than the need for additional technical training in 
fields related to PPB analysis, the study found that many partici- 
pants felt the course should be less theoretical and more practical. 
Our decision to shorten the course related to this conclusion. We 
have carefully reviewed the course to prune out that material which 
is in fact not particularly useful to Federal managers as repre- 
sented by our students. For example, the chance that game theory 
will be actually used as a problem-solving approach by that portion 

of the Federal population represented in the 3-week course is remote 
indeed. 

Secondly, we have de-emphasized material which v*hile poten- 
tially of immediate use could be treated only so superficially in a 
3-week format as to make it seem either useless or uosatisfyingly 
introductory. 
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Many of the statistical approaches fall In this category. By 
improving case teaching techniques and culling weak content, we 
now believe the course can be taught successfully in a 2-week 
format. We will therefore try it in this form on an experimental 

basis. 

When considering the question of balance between theo- 
retical and practical it should be noted that there is nothing 
impractical about theory. The purpose of careful observation and 
consideration of particular events is to permit the formulation of 
a general theory that adequately explains a class of phenomena. 

Thus future events can be predicted, anticipated, and within 
the limits of our knowledge controlled. Man without theory is a 
trained beast. Therefore, we cannot in good conscience extract 
from the PPB course those sections dealing with the theoretical under- 
pinnings of PPB. At the same time we are aware that if the theory 
is not perceived as practical by the students it will be neither 
learned nor applied. 

We are, in consequence, making every effort to relate the theory 
directly and immediately to actual government applications. 'When the 
course started no case studies illustrating PPB applications were 
available. The first case material was introduced in the fourth 
three week course, after development of ten specially prepared teaching 
cases drawn from actual experience. We now have 32 cases available and 
are able to select material that will verify the practical nature of 





the theory. 
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An early problem with the use of case material was the 
lack of experienced case instructors. This problem has been 
largely resolved by two means. First, we have worked carefully 
with regularly employed academic personnel to enhance their 
ability to employ the specialized teaching techniques required 
by the case study method. Second, the center staff have become 
proficient in teaching a number of the cases and can substantially 
supplement the less experienced academic staff. In addition to 
teaching cases, we have secured from BOB, actual program memoranda 
which are discussed sequentially by those in the Agency who pre- 
pared them and those in BOB who reviewed them. 

The study produced a large volume of additional information, 
much of it germane to the complex question of selecting from among 
Federal employees those who might best be sent to particular training 
courses. For example, the evidence that age or length of expert - 
ence, j>er se, made little difference in reported results of the 
training shows that in this case these factors would not have been 
valid predictors, and suggests that we should be cautious about these 
as criteria when advising Agencies whom to nominate to similar courses. 

The hypothesis suggests itself that participant interest and 
need -to -know, arising from new and difficult assignments, can over- 
whelm secondary factors such as age and length of service in deter- 
mining whether employees profit from a relevant learning opportunity. 

In all, the study has become extremely helpful in planning 
curriculum development over approximately a two year period. 




PART B 



Method of Study 

Presentation of Results and Analysis 
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METHOD OF STUDY 

A questionnaire (see Appendix B) was used to elicit infor- 
mation about age, occupation and education; to ascertain attitudes 
toward and applications of course content through limited -response 
questions; and to gather information about needs for further 
training and general reactions through open-end questions. 

To insure that all who answered the questionnaire had been 
back on the job long enough to apply any training results before 
being asked to report its usefulness, only those persons who attended 
the course prior to July 1, 1967, were contacted. Questionnaires 
were sent to 653 former participants, representing 90 percent of the 
722 individuals who attended the 15 sessions of the course from its 

• k 

inception in February 1966 through June 1967. The questionnaires 
were mailed in January 1968; all addresses had been in their Agencies 
at least 6 months after training. 



* k 

Those not contacted were excluded due to lack of sufficient infor- 
mation to construct a mailing address, or because they were employed 
by a government organization other than Federal and consequently 
would be unable to answer many of the questions meaningfully. There 
ware 11 individuals in latter category. 
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SnvMty-eight percent responded. Since the study covers 
results of sessions given over a 17 month period, attention was 
g Iven to the frequency of response from those attending each 
session. The mean response was 79 percent with a range of 66 
percent to 93 percent, and responses above the mean from eight 
classes. Differences in percentage of response seemed to stem 
from chance factors rather than from class priority or passage of 
time. 

Responses were manually tabulated. In addition to totaling 
responses to each question, the responses were also classified by 
the following factors: 

Age 

General Schedule Grade 
Level of Education 

Years of Federal Government employment experience 
Time of course attendance 
Program area 
Occupation 

Working directly or not working directly in PPB 
For comparison, the subdivisions of the classification were 
drawn in the following manner: 

Age: 

Significantly younger (below minus one standard 
deviation from the mean) 

Significantly older (above plus one standard 
deviation from the mean) 
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General Schedule Grade: 

GS -12 and below 
GS-16 and above 

* 

Level of Education: 

Less than bachelor’s degree 
Bachelor's degree 

1. scientific or technical emphasis 

2. business administration or public 
administration 

3. liberal arts (exclusive of above) 

Advanced (more than 30 semester hours 
study beyond a bachelor’s degree) 

Years of Federal Government experience: 

Five years and under 

Thirty years and over 

Time of course attendance: 

First three offerings beginning February 1966 

Last three offerings prior to July 1, 1967 

Program area (as used by the Bureau of the Budget) 

Management 

Natural Resources 

Human Resources 

Science and Technology 

Defense 

International 



'M 

The terms used to describe levels of education and areas of subject 
matter concentration are particular to this study and do not relate 
to actual baccalaureate or other degrees. 



o 

ERIC 



36 



- 32 - 



Former participants working directly in PPB 
Former participants not working directly in PPB 
Occupation: 

Finance (including budgetary functions) 

Staff Support (excluding budget and finance) 

Line Management 
Technical 

Responses were tabulated for each subdivision and the results 
were compared among the subdivisions. 

The Chi-square method was used to test the statistical 
significance of differences found between averaged responses from 
different groups. 
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PRESENTATION OF RESULTS AND ANALYSIS 



I. TOTAL RESPONSE 

#1. ARE YOU CURRENTLY WORKING DIRECTLY IN SOME PHASE OF PPB? 

CHART 1-1 

RESPONSE 




Those who indicated they were working in PPB ware asked to tell 
if their work was principally in planning, in programming, or in 
budgeting, and then to categorize their positions as managerial or 
analytical. Two developments precluded clear interpretation of the 
answers to these questions: (1) Since work in planning, programming, 

and budgeting did not prove to be mutually exclusive activities, the 
number of responses to this question exceeded the number working in 
some phase of PPB; (2) the respondents found it difficult to make 
distinctions between managerial and analytical positions, so compromise 
answers produced a total in excess of the number of respondents. 

Because some of the questions intended only for the people who 
were directly involved in PPB were also answered by those not in this 
group, we had to ignore the occupational distinction and calculate the 
response distribution using the total number of replies. 
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The results indicate that 36 percent work in planning; 30 percent in 
programming; 29 percent in budgeting. Four percent did not respond. 

To the question, "Is your work generally managerial or analytical?" 
the response was: managerial, 33 percent; analytical, 38 percent; no 

response, 29 percent. 

Those who were not currently working in some phase of PPB were 

asked if their work was indirectly supportive of PPB functions. Once 

again the totals exceeded the number in the "not in PPB" category. 

The response distribution was: Yes, 37 percent; and No, 15 percent, 

from all respondents. It can be safely said that about 85 percent were 

working either directly or indirectly in support of PPB. 

#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
PROFESSIONAL BENEFIT TO YOU? 

CHART 1-2 

RESPONSE 



Analytic Techniques 

Economic Concepts 

Program Structuring 

Quantitative Techniques 

Information System Concepts 

Program Memoranda S Program 
St Financial Plan Preparation 

Other 

None 




50 75 

PERCENT 



100 



0 
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No rank ordering or maximum number of selections was 
requested. This question, like all the succeeding ones, did not 
require the distinction of "in PPB" or "out of PPB," thus the 
percentage of response values was calculated on the total number 
of returned questionnaires. Each percentage value should be con- 
sidered independently on a 100 percent scale. Course elements not 
listed on the questionnaire and cited as being of professional 
benefit, included general background information and sources of 
technical information. 

#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EXTENT DID YOU UNDERSTAND 
HOW THE CONCEPTS AND ELEMENTS LISTED IN QUESTION #2 FIT TOGETHER 
TO FORM THE PPB SYSTEM? 



CHART 1-3 



RESPONSE 



Good Understanding 
Fair Understanding 
Didn’t Really Understand 





0 



2 > 



50 

PERCENT 



75 
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#4. DID THE ECONOMIC CONCEPTS 
ABOUT GOVERNMENT PROGRAMS 



RESPONSE 



INFLUENCE YOUR WAY OF THINKING 
IN TERMS OF: 

CHART 1-4 



Outputs 

Costs 

A1 cernat ives 
Objectives 
Ocher 
None 




PERCENT 



The response distribution to this question was calculated 
independently on a 100 percent scale. 
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#5. PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THE QUANTITATIVE 
AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN HELPFUL TO YOU. 



CHART 1-5 



RESPONSE 




Other applications mentioned included performance evaluation, 
documentation, and establishing realistic goals. 
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#6. TP YOJ WERE GOING 10 ATTEND THE COURSE NOW, IS THERE ANY 
PORTION WHERE YOU FEEL THE EMPHASIS SHOULD BE CHANGED? 

TABLE 1-6 



PORTION 


INCREASE 

or 

/o 


DECREASE 

% 


NO CHANGE 

% 


Preparation of Program 
Memoranda and Program 
& Financial Plan 


46.5 


7.9 


29.7 


Program Structuring 


45.5 


11.3 


33.9 


Systems Analysis 


44.4 


5.9 


28.3 


Case Studies 


40.4 


15.2 


32.3 


Information Theory 


31.1 


12.5 


38.0 


Ma thema tics/ Statistics 


19.2 


36.6 


29.5 


Economics 


17.6 


25.1 


42.8 


Other (including "practical 


7.9 







examples," "Bureau of the 
Budget use of PM," the 
budget process," and 
political implications.") 

This question was asked on the premise that the most valuable 
critique of the treatment of subject matter in a training program may 
be that provided by former participants who have had an opportunity to 
assess their day-to-day needs and compare these needs with the concepts, 
skills, and techniques presented through the course. 

The response distribution displayed above is ranked according to 
those portions seen as needing additional emphasis. Totals do not 
equal 100 percent due to multiple replies and no comment by 
respondents on some portions. 
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#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT YOU HAVE 
CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH OUTSIDE COURSES, 
PERSONAL READINGS, ETC.? 



CHART 1-7 



RESPONSE 




Those who answered "yes" to this question were asked to list 
the subjects of their study. The subjects mentioned fell into seven 
general categories listed here in order of their frequency: 

Analytic Techniques 

Planning, Programming, and Budgeting as as 
integrated concept 

Economics 

Numerical Science 

Personal Reading (subject unspecified) 

Management and Administration 

Other (including no response to question 
about subject matter) 
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The table below shows the subjects within each category, the 
number of times each subject was mentioned, and the percentage 



response distribution for each category: 



TABLE 1-7 


CATEGORY SUBJECT 


FREQUENCY 

LISTING 


Analytic Techiiques 

Cost /Benefit 


Analysis 


18 


Information System 


21 


Program Analysis 


6 


Systems Analysis 


40 


Operations Research 


3 


Planning, Programming, 
and Budgeting as an 
integrated concept 


PPB (General) 


59 


Budgeting 


18 


Program Structure 


7 


Financial Planning 


6 


Economics 
Numerical Science 


Statistics 


30 


Mathematics 


15 


Linear Programming 


2 


Game Theory 


2 


Personal Reading (Subject not specified) 
Management and Administration 


Management 


17 


Public Administration 


8 



Other 



PERCENT OF 
POSITIVE RESPONSES 

31.7 



30.7 



28.0 

16.7 



9.2 

8.5 



7.8 
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#8. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A 
DIRECT RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 

CHART 1-8 

RESPONSE 



Positive 



Negative 



No Response 



0 25 50 75 100 

PERCENT 



The outputs described fell into three general classes: (1) 

analysis; (2) preparation of program memoranda and financial plans 
and related documents; and (3) communication and training. 

Examples of analysis include: 

"The seminar . . . was oi value not because 
it was a handbook in how to apply PPB, but 
because it demonstrated ar» approach, a way 
of confronting problems and questions. This 
was the most valuable contribution the PPB 
seminar could have made.". 

"Output statistics were fairly well established 
prior to attendance at course. However, 
thought provoking questions, prompted by such 
attendance, were raised at PPB review sessions 
so that suggested additional outputs are being 
studied for future use." 

"I now have a good background of PPB theories 
and principles which I almost automatically 
apply to daily work situations. In fact, my 
whole pattern of solving problems has been 
influenced by my experience with PPB." 
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"Anaiysis of management costs of R&D 
operations." 

"Use of the knowledge gained in structuring 
analytical outlines to be completed by 
agencies in justifying their proposals. 

Also provides a basis for improved review 
of such analyses after their submission. 1 ' 

"Have been inspired to improve progressively 
the techniques employed in quantitative 
rationale, not only in the total as regards 
(activity) components, but also in a com- 
plicated breakdown or distribution technique 
including "initiatives," "cost/benefit," 

"regional," and other distributions. For 
clarity, quantification and output are 
synonomous dealing with foreigners trained, 
oriented, and/or serviced." 

Examples of preparation of program memoranda and financial plans 

"50-page FDA staff paper on user charges, em- 
phasizing sunk costs vs. marginal costs (a 
concept I did not understand until I took 
the course), was well received and instru- 
mental in decision not to implement user 
charges on certain FDA services." 

"Program projections involving population, 
income, construction of medical facilities, 
medical schools and graduates into the medical 
field, manpower, etc." 

"PFP for my agency." 

"Structuring my division's research program." 

"We (the CAB) presented our budget to BOB 
on a program basis and also to Congress. The 
budget hearing with the House sub-committee 
indicated a high degree of satisfaction with 
the program approach on the budget." 
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"Wrote program memoranda for the program 
elements accounting, personnel, management 
inspection and audit, and administrative 
service. " 

“My outputs are primarily budget narratives, 
explanations and analyses. These have been 
in budgeting toward relating program needs 
to program objectives and achievements. The 
seminar helped by relating the budgetary 
activity to the longer range planning and 
programming aspects of PPB." 

'•Development of interpretive material on 
agency budget development." 

"Report structure of 0E0 Financial System 
is now on a PPB basis." 

"Preparation and review of Bureau’s Program 
Memoranda. " 



Some representative outputs in the area of communication and training 



include: 



"I had the opportunity to discuss with field 
level personnel how they viewed the current 
workability of the current output measures 
in the agency in which they work and for 
which 1 am responsible in BOB. Action as a 
direct result of this is still underway." 

"Acceptance by NASA General Management of 
display and review techniques that focus on 
implications of decisions made NOW as against 
decisions that do not have to be made until 
NEXT tfEAR or NEXT YEAR plus 1." 
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"1 an suggesting and obtaining greater 
participation of line supervision in the 
preparation of financial plans. We are 
much more conscious of the cost /benefit 
factor. M 

"Provided specialized staff service in 
conduct of PPBS Seminar for Metropolitan 
Kansas City sponsored by local F.E.B. Chapter 
on January 8, 1968." 

"I am currently attempting to develop a 
presentation to top management as to what 
they need to know to make decisions - 
primarily how to establish priorities for 
funding , cost/benefit ratios, incremental 
costs, etc," 

"I have since given several lectures to 
various interested groups in my former 
organ izat ion." 

"I was asked to give a talk which explained 
the seminar, defined terms, and gave examples 
of use of PPB." 

"Based on the course, I had my attitudes on 
the importance of objectives substantiated. 
With this buttress to my confidence, I 
was able to convince our program oriented 
planning staff that they should concentrate 
on objectives and on alternatives that 
should be considered in achieving these 
objectives." 
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II. AGE 

Although the total response to the questionnaire was favorable 

and generally in line with the expectations of Bureau of Training 

personnel involved in the PPBS Seminar, it appeared relevant to 

investigate the impact of distinctive characteristics of the former 

participants on the value of the course to any given individual. The 

question of age and response to the course was approached by separating 

from the group the response of those who are significantly younger 

(35 years of age and under) and those significantly older (54 years of 

* 

age and older) . 

Three questions were selected as being most susceptible to in- 

f luence by the age of the respondent. The questions selected for this 

analysis and the responses are shown in the tables below: 

#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EXTENT DID YOU UNDERSTAND 
HOW THE CONCEPTS AND ELEMENTS LISTED IN QUESTION #2 FIT TOGETHER 
TO FORM THE PPB SYSTEM? 





TABLE II-l 




RESPONSE 


YOUNGER 


OLDER 




(n=93) 


(n=74) 




% 


% 


Good Understanding 


53.1 


52.7 


Fair Understanding 


38.7 


43.2 


Didn't Really Understand 


3.2 


4. 1 



The standard deviation in the age distribute ion of all those returning 
the questionnaire was 8.7 years with the mean age 44.7 years. For 
purposes of this study, significantly younger and significantly older 
are those ages in the distribution that fall beyond plus one and minus 
one standard deviation. 
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While there appears to be an edge of difference between 
these two groups in favor of the younger in their comprehension 
and appreciation of the conceptual scheme of PPB, actually the 
difference was shown by statistical test not to be significant, 
or reliable for predictive purposes. 

#4. DID THE ECONOMIC CONCEPTS INFLUENCE YOUR WAY OF THINKING ABOUT 
GOVERNMENT PROGRAMS IN TERMS OF: 



TABLE I I- 2 



RESPONSE 


YOUNGER 


OLDER 




(n=93) 


(n=74) 




% 


% 


Yes - Outputs 


79.7 


56.8 


Yes - Costs 


59.1 


59.5 


Yes - Alternatives 


78.5 


81.1 


Yes - Objectives 


68.7 


69.8 


Yes - Other 


5.4 


6.8 


No Yeses 


7.5 


10.8 



(The columns do not equal 100 due to multiple replies.) 

This analysis does not produce any information that can lead 
to a conclusion that age is a determining factor in the receptivity 
of participants to accept new approaches to familiar challenges. 



Additional tests of significance did not produce measurable differences 

#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT YOU HAVE 
CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH OUTSIDE COURSES, 
PERSONAL READINGS, ETC.? 



TABLE I 1-3 



RESPONSE 


YOUNGER 


OLDER 




(n=93) 


(n=74) 




% 


% 


Yes 


69.9 


63. 5 


No 


29.0 


36. 5 


No Response 


1.1 





No significant difference. 

- 51 
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III. GENERAL SCHEDULE GRADE 

Since 467 of the 508 respondents were in positions under 
the General Schedule, no attempt was made to equate grade classi- 
fications under other systems to GS grades. Of those under the 
General Schedule, 82 percent were in grades 13, 14, and 15 at the 
time they completed the questionnaire. Recognizing that the responses 
of this group would have appreciable effect on the overall result, the 
question "Is there any significant difference in the responses of 
those in grades above and below this interval ?" was raised. Five 
questions that might be affected by grade were selected for analysis. 
These questions and the responses are shown in the tables below: 

#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
BENEFIT TO YOU? 



TABLE III-l 



RESPONSE 



GRADES 12 AND UNDER 
<n=54) 

% 



GRADES 16 AND OVER 
(n=28) 

% 



Economic Concepts 
Quantitative Techniques 
Analytic Techniques 
Information System Concepts 
Program Structuring 
PM and PFP Preparation 



51.9 
24. 1 
48. 1 
31.5 

48. 1 

24. 1 



63.0 
33.3 

37.0 

37.0 
40.7 

25.0 



(Columns do not equal 100 due to multiple replies.) 



ERiC 



52 



- 48 - 



A1 though the differences in the responses between the groups 
are not statistically significant, they give some thin support to 
the premise that training in conceptual areas is more valuable to 
higher level employees while instruction in skills and techniques 
has greater utility for those in lower grades. 

#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EXTENT DID YOU UNDERSTAND 
HOW THE CONCEPTS AND ELEMENTS LISTED IN QUESTION #2 FIT TOGETHER 
TO FORM THE PPB SYSTEM? 



TABLE II 1-2 



RESPONSES 


GRADES 12 AND UNDER 


GRADES 16 AND OVER 




<n=54) 


(n=28) 




% 


% 


Cood Understanding 


46.3 


40.7 


Fair Understanding 


50.0 


55.6 


Didn't Really Understand 


3.7 


3.7 



The slightly higher degree of understanding reported by those 
in lower grades is not significantly different from those in higher 
grades or from the response of the total group. 



Chi-square tests indicated a high probability that these two groups 
are from the same population in terms of this question. 
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#5. PLEASE INDICATE THE SITUATIONS WHERE APPLICATION OF THE 
QUANTITATIVE AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN 
HELPFUL TO YOU. 



TABLE III-3 



RESPONSE 



Problem Structuring 
Problem Analysis 
Problem Solving 
Communicating with Others 
Identifying and Considering 
Alternatives 



GRADES 12 AND UNDER CRADES 16 AND OVER 



(n=54) 

% 

22.2 

48.1 

20.4 

46.6 

59.3 



(n=28) 

33.3 

40.7 

18.5 

48.1 

63.0 



(Columns do not total 100 due to multiple replies.) 



Here again, the marginal differences between groups are not 
significant. Both response patterns are also parallel with the answers 
from all the respondents. 

in. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT YOU HAVE 
CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH OUTSIDE COURSES, 
PERSONAL READINGS, ETC.? 



TABLE II 1-4 

RESPONSE GRADES 12 AND UNDER GRADES 16 AND OVER 



(n=54) (n-28) 

% % 

Yes 57.4 63 *° 

No 37.0 37.0 

No Response 5.6 



No significant difference. 
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#8. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A 
DIRECT RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 



RESPONSE 



Yes 

No 

No Response 



TABLE I I I- 5 



GRADES 12 AND UNDER 
(n=54) 

% 



GRADES 16 AND OVER 
(n=28) 

% 



24.1 37.0 

70.4 63,0 

5.5 



While those in higher grades more frequently cited outputs, the 
differences did not meet the statistical test of significance. 
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IV. EDUCATION 

The responses were aivlded Into the five education 
categories described in "Method of Study": less than bachelor's; 

bachelor's degree in science or technology; bachelor's degree in 
business administration or public administration; bachelor's degree 
in liberal arts, excluding the forementioned; and advanced, in- 
cluding more than 30 hours study beyond a bachelor's degree. The 
responses were analyzed separately on the assumption that a partici- 
pant's level of edication as well as his concentration of courses 
may have significant influence on his ability to assimilate the 
training and to apply it to his duties. 

Eighty percent of the respondents hold bachelor's degrees or 
more. Bachelor's degree holders who have not earned a master's degree 
or its equivalent comprised the largest segment of the total group 
answering the questionnaire (48.5 percent). Those with advanced 
education were next (31 percent). The frequency of types of edication 
reported by those at the bachelor's level was bachelor's degree in 
business or public administration (19.1 percent), bachelor's degree 
in a scientific or technical area and bachelor's degree in liberal 
arts other than business or public administration, occurring with 
equal frequency (14.7 percent) each. Those with less than a bachelor's 
degree included 15.1 percent of all respondents. Five questions were 

selected for analysis. 
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Piv* quest inns w re selected as b.'ing relevant to <*du- 
ralit».ial If-Vr!. has* questions ami the responses an* shown in 
t 'ii- following tables! 

WHA ! I .l.I.i-tl-.N I OR I’.l.KMKNIS OF THK OUJRSK dAVK 8KKN OF URKATFST 
FROFKSSIONAI. BI'.N'KFIT TO YOU? 

TABi.K IV- I 



r-:sp:>nsk 


NO 

Oir.RKK 
( n-77) 


BVrdKLOR 
OF SCJKNCK 
(n-75) 

Z 


backklor 

OF BA OR FA 
(n-9?) 

7, 


BACHK1.0R 
OF ARTS 
(n=75) 

**t 


ADVANCED 

EXJUIV. 

(n=15H> 


* 

T0TAI. 

( Ai.L) 
(n=50K) 

7 


Mi onomic (*«Mc**pts 


44.2 


bb . b 


43.3 


52.0 


45. b 


48.7 


Qa ant. f t at fvc Techniques 


23.4 


34. 7 


30.9 


2;3 . 7 


23. 1 


31 . 3 


Analytic IVchniqucs 


57.1 


5b. 0 


4H. 5 


64.0 


51.3 


50.3 


Information Systom (Vmccpt s 


27.3 


21.3 


34.0 


32.0 


32. 3 


29 . 3 


Program Structuring 


45.5 


37. 3 


41.2 


35.0 


42.4 


42.8 


PM* PFP Preparation 


23.4 


17.3 


25. B 


24.0 


20. 3 


22.0 


Pther 


5.2 


8.0 


10.3 


6.7 


12.0 


H. 3 



nht* total responses are repeated for comparison.) 

iho responses Were not significantly different except in the 
professional benefit of economic concepts. Here the respondent s with 
a bachelor of science degree proved to be significantly different 
from all th - other groups except those having a bachelor's degree in 
liberal arts. The difference was present to the extent of less than 
one percent probability of similarity. 



* 

Percentages under * All* do not eqjal an average for the other categories 
because we could not determine education classification for some 
quest ionnaires. 
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#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EiCFENT DID YOU 

UNDERSTAND HOW THE CONCEPTS \ND ELEMENTS LISTED IN QUESTION 
#2 FIT TOGETHER TO FORM THE PPB SYSTEM? 







TABLE IV- 2 










RESPONSE 


NO 

DEGREE 

(n=77) 

7. 


MUmOR 
OF SCIENCE 
(n-75) 

t 


BACHELOR 
OF BA OR PA 
(n-97) 

Z 


BACHELOR 
OF ARTS 
(n~?5) 

% 


ADVANCED 

EQUIV. 

(n-=158) 

■7 


TOTAL 
(Al.D 
( n= 508 ) 

7 


Good Understanding 


41.6 


44.0 


59.8 


56.0 


58.9 


50.9 


Fair Understanding 


53.2 


52.0 


38.1 


41.3 


37.3 


43.0 


Didn't Really Understand 


2.6 


4.0 


2.1 


2.7 


3.8 


2.4 


No Response 


2.6 


- 


- 


- 


- 


3.7 



Ihe frequency of optimum response correlates positively with 
academic backgrounds that are more likely to have included PPB related 
subject matter and with the number of years of education. A 
statistically significant difference at the 95 percent confidence 
interval occurred between the response of those with no degree and 
those with a bachelor of business administration or public administration, 
or a bachelor of arts, or a master's degree. A significantly smaller 
oercentage of the bachelor of science group indicated a good under- 
standing than of the bachelor of business or public administration and 
the master's degree groups. 
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#4. DID THE ECONOMIC CONCEPTS INFLUENCE tfOUR WAY OF THINKING 
ABOUT GOVERNMENT PROGRAMS IN TERMS !)?: 

TABLE IV-3 



YES RESPONSE 


NO 

DEGREE 

(n=77) 

X 


bachelor 
of SCIENCE 
(n-75) 

% 


BACHELOR 0? 
BA OR PA 
(n=97> 

f 

> 


BACHELOR 
0? ARTS 
(n-75) 

4 


ADVANCED 

(n~l58j 

S 


TOTAL 

(ALL) 

' n=50P 
4 


Outputs 


62.3 


68.0 


62.0 


66.6 


64.6 


64.7 


Ci.cts 


68.8 


66.7 


61.9 


62.7 


60.1 


62.8 


Alternatives 


83.1 


82.7 


; 4.2 


74.7 


7*. 7 


76.4 


Objectives 


67.5 


73.3 


64.9 


69.3 


60.1 


64.5 


Other 


2.6 


10.7 


5.2 


7.8 


5 0 


5. 7 


No Yeses 


3- ? 


3.4 


A. 2 


9. 3 


10. 1 


5.3 


No Response 


1.3 




2.1 









No significant difference. Marginally greater responses nay be 
due to the lack of PPB related subjects in certain academic programs. 
For instance, the relatively higher influence of economic concepts on 
the thinking of "no degree" and bachelor of science respondents about 
alternatives might be explained by less previous formal instruction in 
economics. 
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#5 PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THE QUANTITATIVE 
AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN HELPFUL TO YOU. 

TABLE IV-4 

NO BACHELOR BACHELOR OF BACHELOR 



RESPONSE 


DECREE 

(n=77) 

Z 


OF SCIENCE 
(n=75) 

■7 

•O 


BA OR PA 
(n=97) 

7. 


OF ARTS 
(n=75) 


ADVANCED 

(n=158) 

■7 

•o 


(ALL) 

(n«303) 

■7 


Problem Structuring 


15.6 


29.3 


21.6 


38.7 


34.8 


29.1 


Problem Analysis 


53.2 


52.0 


44.3 


52.0 


53.2 


49.7 


Problem Solving 


35.1 


21.3 


22.7 


30.7 


34.8 


23 3 


Communicating with 
Others 


45.5 


49.3 


46.4 


48.0 


43.0 


43.2 


Identifying and 
Considering 
Alternatives 


67.5 


61.3 


51.5 


58. 7 


58.2 


58.0 


Other 


2.6 


5 3 


$.* 


5.3 


4.4 


4.6 


No Response 


7.8 


5.3 


4 1 


4.0 


7.6 


5 8 



This table reveals Interesting distinctions among the education 
groups. There is a significantly lower percentage of positive responses 
about problem structuring by the no degree group when compared with all 
the other groups except bachelor of business or public administration. 

The no degree group was also significantly lower than bachelor of science 
group and the bachelor of public administration group in the application 
of quantitative and systems analysis techniques to problem solving. 
Identifying and considering alternatives provided another significant 
distinction between the no degree group and the bachelor of public admini- 
stration group, the no degree group being the lower. All the distinctions 
were made using a 95 percent confidence interval. 




* 
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#7. DID THE COURSE STIMULATE Yu» - r •: iREST TO THE EXTENT THAT YOU 
HAVE CONTINUED TO STUDY IN RELAXED SUBJECTS THROUGH OUTSIDE 
COURSES, PERSONAL READINGS, ETC.? 



TABLE IV- 5 



RESPONSE 


NO 

DEGREE 

(n=77) 

% 


BACHELOR 
OF SCIENCE 
(n=75) 

% 


BACHELOR 
BA OR PA 
(n=97) 

% 


BACHELOR 
OF ARTS 
(n=75) 

% 


ADVANCED 

(n=158' 

% 


TOTAL 

(ALL) 

(n=503) 

% 


Yes 


AS. 2 


68.0 


49.5 


65.3 


66.5 


58.0 


No 


49.2 


29.3 


49.5 


30.7 


33.5 


35.8 


No Response 


2.6 


2.6 


1.0 


4.0 




6.2 



The percentage of positive responses of those with no 
degree and those with a bachelor of business or public admini- 
stration degree are significantly lower than the other groups and 
than the total response. 



O 
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V. YEARS OF GOVERNMENT EXPERIENCE 

To assess the influence that length of service and cumulative 
experience might have on the reactions of participants, the answers 
from those with 5 years or less experience, and from those with 30 
or more years were compared. The mean of the experience distribution 

is 18.10 years, the median 19 years. 

The questions selected for comparison and the responses are shown 

in the following tables: 

#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
PROFESSIONAL BENEFIT TO YOU? 

TABLE V-l 



RESPONSE 


5 YEARS AND UNDER 
(n=40) 

% 


30 YEARS AND OVER 
(n=39) 

X 


TOTAL 

(n=508) 

% 


Economic Concepts 


37.5 


64.1 


48.7 


Quantitative Techniques 


15.0 


28.2 


31.3 


Analytic Techniques 


42.5 


71.8 


50.3 


Information System Concepts 


35.0 


25.6 


29.3 


Program Structuring 


42.5 


41.0 


42.8 


PM, PFP Preparation 


12.5 


10.3 


22.0 


Other 


17.5 


2.6 


8.3 


No Response 


- 


2.6 


- 



(The total responses are repeated for comparison.) 
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The significant differences that occurred pertain to 
activities that are primarily conceptual in nature and favored 
the group with more experience. The frequency of selection of 
Economic Concepts as a course element that had been of pro- 
fessional benefit by the junior group was significantly less than 
this selection by the senior group. The same is true about the 
selection of analytic techniques, and in both cases the senior 
group was significantly greater than total. There was also a 
significant difference in the frequency of "other” elements 
between the junior and senior groups. 

Some of "other” elements mentioned by the junior group 
included "placing PPB in context," "prestige value of the course,' 
"organizational problems," and "reading material." 

#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EXTEN 1 OID YOU 

UNDERSTAND HOW THE CONCEPTS AND ELEMENTS LISTED IN QUESTION 
//2 FIT TOGETHER TO FORM THE PPB SYSTEM? 



TABLE V -2 



RESPONSE 5 YEARS AND UNDER 

( n=40) 

% 


30 YEARS AND OVER 
(n=39) 

7. 


TOTAL 

(n= 508 ) 

% 


Good Understanding 


67.5 


53.8 


50.9 


Fair Understanding 


30.0 


43.6 


43.0 


Didn't Really Understand 


2.5 


- 


2.4 


No Response 


- 


2.6 


3.7 




63 



While marginal differences favored the group with 5 years 
experience and under } there is no statistically significant 
difference in the responses. 

#5. PLEASE INDICATE THE SITUATIONS WHERE APPLICATION OF THE 

QUANTITATIVE AND SYSTEM ANALYSIS TECHNIQUES HAVE BEEN HELPFUL 

TO YOU. 



TABLE V-3 



RESPONSE 5 YEARS AND UNDER 

( n=40) 

% 


30 YEARS AND OVER 
(n=39) 

% 


TOTAL 

(n=508) 

% 


Problem Structuring 


37.5 


17.9 


29.1 


Problem Analysis 


55.0 


46.2 


49.7 


Problem Solving 


22.5 


30.8 


28.3 


Communicating with Others 


30.0 


38, :> 


43.2 


Identifying and Considering 
Alternatives 


64.0 


56.4 


58.0 


Other 


2.5 


2.6 


2.5 


No Response 


5.0 


7.7 


2.0 



There is a significant difference between responses of the 
junior and senior groups about the application of quantitative and 
systems analysis techniques to problem structuring. The junior 
group was higher. No other significant differences we? e detected. 
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#4. DID THE ECONOMIC CONCEPTS INFLUENCE YOUR WAY OF THINKING 
ABOUT GOVERNMENT PROGRAMS IN TERMS OF: 

TABLE V-4 



RESPONSE 


5 YEARS AND UNDER 
(n=40) 

% 


30 YEARS AND OVER 
(n=39) 


TOTAL 

(n=508) 

'O 


Outputs 


62.5 


56.4 


64.7 


Costs 


60.0 


51.3 


62.8 


Alternatives 


80.0 


79.5 


76. 4 


Objectives 


70.0 


69.2 


64. 5 


Other 


2.5 


5.1 


5.7 


No Yeses 


12.5 


5.1 


5.3 



The patterns of responses by both groups were essentially 
parallel. 

#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT YOU HAVE 
CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH OUTSIDE COURSES, 
PERSONAL READINGS, ETC.? 

TABLE V-5 



RESPONSE 


5 YEARS AND UNDER 
(n=40) 

% 


30 YEARS AND OVER 
(n~39) 

% 


TOTAL 

(n=508) 

% 


Yes 


67.5 


59.0 


58.0 


No 


32.5 


41.0 


35.8 


No Response 


- 




6.2 



While there is a higher frequency of further study by the iunior 
group, the difference is not significant. It is perhaps noteworthy 
that over half of the senior experience group as well as the older age 
group individually sought additional information. 
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#8. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A 
DIRECT RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 

TABLE V-6 



RESPONSE 


5 YEARS AND UNDER 
( n=40) 

% 


30 YEARS AND OVER 
(n=39) 

% 


TOTAL 

(n=508) 

% 


Yes 


30.0 


35.9 


31.9 


No 


70.0 


64.1 


62.0 


No Response 


- 


- 


6.1 



The factor of experience had no apparent influence on answers 
to this question. 



#10. FOLLOWING YOUR EXPERIENCE WITH THE COURSE, HAVE YOU RECOMMENDED 
IT FOR OTHER IN YOUR AGENCY? 

TABLE V-7 



RESPONSE 


5 YEARS AND UNDER 
( n=40) 

% 


30 YEARS AND OVER 
(n=39) 

% 


TOTAL 
( n=508 
% 


Yes 


87.5 


74.4 


80.6 


No 


12.5 


23.0 


15.0 


No Response 


- 


2.6 


<.4 



No significant difference. 
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VI. COMPARISON OF RESPONSES OF THOSE ATTENDING THE FIRST THREE 
SESSIONS OF THE PROGRAM WITH THE RESPONSES OF THOSE 
ATTENDING THE LAST THREE SESSIONS COVERED 3Y THE SURVEY. 

In the seventeen months between February 1966 and July 1967 

when the fifteen seminars considered by this survey were conducted, 

much growth took place in the knowledge about the application of 

PPB to civilian agencies of the Federal Government. A continuing 

effort has been made to incorporate new refinements and perceptions 

into the seminars to make them more relevant to the needs of the 

participants and their employing Agencies. To evaluate the success 

of changes in seminar design and content in meeting the needs of 

the participants, we compared responses of those attending the 

initial three seminars from February through August 1966 and those 

attending the latter segment of the FY 1966-67 series, April through 

June 1967. The relevant questions and the responses are shown in 

* 

the following tables. 

#1. ARE YOU CURRENTLY WORKING DIRECTLY IN SOME PHASE OF PPB? 



TABLE VI- 1 



RESPONSE 


SESSIONS 1,2, & 3 
(n»85) 

% 


SESSIONS 13,14, & 15 
(n-=93) 

% 


CHANGE IN PERCENT 
AGE POINTS 

Z 


Yes 


71.6 


59.1 


-12.6 


No 


25.9 


40.9 


+15.0 


No Response 


2.4 







* 

A statistically significant difference In the response Is present when 
the percentage point change is 14 or greater. 
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By this criterion the class population in the latter 
sessions was significantly different. Most participants in the 
early sessions were selected because they were to be directly 
involved in the operation of the PPB system. Later groups 
included more line managers and staff personnel with specialities 

other than PPB. 



#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
PROFESSIONAL BENEFIT TO YOU. 



RESPONSE 

Economic Concepts 

Quantitative Techniques 

Analytic Techniques 

Information System 
Concepts 

Program Structuring 
PM, PFP Preparation 



TABLE VI-2 



SESSIONS 1,2, & 3 
(n=85) 

% 

54.1 
24.7 

43.5 

27.1 

35.3 

16.5 



SESSIONS 13,14, & 15 CHANGE IN PERCENTAGE 



(n--92> 

49.5 

35.5 
53 ft 

34.4 

51.6 
24 . 7 



POINTS 

% 

- 4.6 
+ 10.8 
+ 10.2 

+ 7.3 
+16.3 
+ 8.2 



The apparent contradiction posed *'y significantly higher 
benefit from Program Structurinf (a PUB oriented concept) by those 
attending the latter session (whicn included significantly more 
people not in PPB) may be explained by th< r-, f moment of the concept 
that took place over the 15 months span. 
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#3. AT THE TIME YOU LEFT THE COURSE, TO WHAT EXTENT DID YOU 

UNDERSTAND HOW THE CONCEPTS AND ELEMENTS LISTED IN QUESTION 
#2 FIT TOGETHER TO FORM THE PPB SYSTEM? 



TABLE VI-3 



RESPONSE 




SESSIONS 1,2, & 
(n=85) 

% 


3 SESSIONS 13,14, & 15 

(n=93) 

Z 


CHANGE IN PERCENTAGE 
POINTS 

% 


Good Understanding 


54.1 


55.9 


+1.8 


Fair Understanding 


42.4 


40.9 


-1.5 


Didn't Really Understand 


3.5 


1.0 


-2.5 




No significant difference. 






#4. 


DID THE ECONOMIC CONCEPTS INFLUENCE YOUR WAY OF THINKING 
ABOUT GOVERNMENT PROGRAMS IN TERMS OF: 






TABLE 


VI-4 




RESPONSE 




SESSIONS 1,2, & 3 
(n=85) 

Z 


SESSIONS 13,14, & 15 
(n=93) 

% 


CHANCE IN PERCENTAGE 
POINTS 

% 


Outputs 




70.6 


69.9 


- .7 


Costs 




60.0 


62.4 


+2.4 


Alternatives 




78. 8 


74.2 


-4.6 


Objectives 




68.2 


64.5 


-3.7 


0*;her 




8.2 


4.3 


-3.9 



No significant difference. 
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#5. PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THE 

QUANTITATIVE AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN 
HELPFUL TO YOU. 



TABLE VI- 5 



RESPONSE 


SESSIONS 1,2, & 3 
<n=85) 

tO 


SESSIONS 13,14, 15 

(n=93) 

£ 


CHANGE IN PERCENTAGE 
POINTS 

X 


Problem Structuring 


34.1 


19.4 


- 14.7 


Problem Analysis 


44.7 


49.5 


+ 4.8 


Problem Solving 


28.2 


23.7 


+ 4.5 


Comnunicating with Others 


50.6 


46.2 


- 4.4 


Identifying and Considering 
Alternatives 


55.3 


57.0 


+ 1.7 



This indicates that the techniques have been significantly 
less helpful in problem structuring to those who attended later, 
but there has also been a marginal shift to emphasis toward analysis 
and selection of alternatives as time passed. 
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#6. TF YOU WERE GOING TO ATTEND THE COURSE NOW, IS THERE ANY 
PORTION WHERE YOU FEEL THE EMPHASIS SHOULD BE CHANGED? 



TABLE VI-6 



RESPONSE 


SESSIONS 1,2, & 3 


SESSIONS 13, 14, & 15 


CHANGE IN 




( n=85) 


(n=93) 


PERCENTAGE POINTS 




% 


% 


Z 


ECONOMICS 


Increased 


20.0 


19.4 


- .6 


Decreased 


23.5 


32.3 


+8.8 


No Change 


42. A 


47.3 


+4.9 


MATHEM AT ICS/ STAT IS T ICS 


Increased 


21.2 


14 0 


-7.2 


Decreased 


42.4 


45.2 


+2.8 


No Change 


22.4 


30.1 


+7.8 


CASE STUDIES 


Increased 


60.0 


40.9 


-19.1 


Decreased 


7.1 


16.1 


+ 9.0 


No Change 


22.4 


36.6 


+14.2 


SYSTEMS ANALYSIS 


Increased 


49.4 


43.0 


- 6.4 


Decreased 


10.6 


3.2 


- 7.4 


No Change 


30.6 


45.2 


+14.6 


INFORMATION THEORY 


Increased 


27.1 


32.3 


+ 5.2 


Decreased 


20.0 


11.8 


- 8.2 


No Change 


34.1 


37.6 


+ 3.5 
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RESPONSE 


SESSIONS 1,2, & 3 
(n=K5) 

7. 


SESSIONS 13,14. {, 15 
(n=93) 

% 


CHANGE IN 
PERCENTAGE POINTS 
% 


PROGRAM STRUCTURING 








Increased 


38.8 


46.2 


+7.4 


Decreased 


1 1 . P 


12.9 


+ 1.1 


No Change 


34.1 


40.9 


+6.8 


PREPARATION OF PM, PFP 








Increased 


50.6 


51.6 


+ 1.0 


Decreased 


10.6 


7.5 


-3.1 


No Change 


25.4 


33.3 


+7.4 



Those who attended last expressed a significantly greater degree 
of satisfaction with case studies and the portion dealing with systems 
analysis. Deficiencies in those areas were recognized by the Commission 
when the series started, and effort was made to strengthen these portions. 



#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT YOU HAVE 
CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH OUTSIDE COURSES, 
PERSONAL READINGS, ETC.? 



TABLE VI- 7 



RESPONSE 


SESSIONS 1,2. & 3 
(n=85) 

% 


SESSIONS 13,14, & 15 
(n=93) 

% 


CHANGE IN 
PERCENTAGE POINTS 

% 


Yes 


64.7 


58.0 


- 6.7 


No 


35.3 

No significant difference. 


40.9 


+5.6 
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#8. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A 
DIRECT RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 

TABLE VI-8 



RESPONSE 


SESSIONS 1,2, & 3 

V 5 > 


SESSIONS 13,14, 6. 15 CHANGE 

(n=93> 

% 


IN PERCENTAGE 
POINTS 

% 


Yes 


30.6 


30.1 


- .5 


No 


67. I 


68.8 


+ 1.7 




No significant difference. 






no. 


FOLLOWING YOUR EXPERIENCE WITH 
IT FOR OTHERS IN YOUR AGENCY? 


THE COURSE, HAVE YOU RECOMMENDED 




TABLE VI- 9 






RESPONSE 


SESSIONS 1,2, & 3 
(n=85> 

% 


SESSIONS 13,14, & 15 CHANGE 
(n=93) 

% 


IN PERCENTAGE 
POINTS 

% 


Ves 


84.7 


78.5 


-6.2 


No 


14.1 


18.3 


+4.2 




No significant difference. 
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VII. PROGRAM AREAS 

To assess the reactions of former participants who are 

working in different types of programs, the questionnaires were 

divided into six groups according to the program area of the 

respondent’s employing agency. The programs are Defense, Human 

Resources, International, Management, Natural Resources, and 

Science and Technology. Typical organizations in each category, 

as listed, are Department of the Army, Office of Economic Opportunity, 

Agency for International Development, Civil Service Commission, 

Department of Interior, and Agricultural Research Service. 

The questions selected as being relevant to this analysis and 

★ 

the responses are shown in the tables below. 

#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
PROFESSIONAL BENEFIT TO YOU? 



The sum does not equal 508 since former participants are presently 
in long term education programs or other assignments not directly 
related to the program area of their employing organization. 
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TABLE VII-1 



RESPONSE 



HUMAN 



DEFENSE RESOURCES INTERNATIONAL M<\NAGEMENT RESOURCES 



NATURAL SCIENCE 





(n=15> 

% 


(n~l 12) 

‘O 


(n=34) 

c* 

/o 


(n=166) 


(n=128) 


(n=50) 

C*t 


Economic Concepts 


53.3 


42.9 


17.6 


49.4 


61.7 


64.0 


Quantitative Techniques 


40.0 


28.6 


23.5 


30.1 


31.3 


40.0 


Analytic Techniques 


60.0 


55.4 


58.8 


54.8 


54.7 


50.0 


Information System Concepts 


33.3 


31.2 


35.3 


32.5 


26.6 


24.0 


Program Structuring 


53.3 


44.6 


44.1 


39.8 


45.3 


40.0 


PM, PFP Preparation 


46.7 


24. 1 


14.7 


22.3 


20.3 


22.0 


Other 


13.3 


8.0 


20.6 


3.6 


7.0 


14.0 



Those in International programs indicated significantly less 
professional benefit from Economic Coreepts than those did in any of 
the other groups. This may be indicative of the difficulty in 
expressing some program objectives in quantitative terras. The more 
frequent mention of PM and PFP Preparation by the Defense program 
category is not significant statistically. 
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1b'-*. DID THE ECONOMIC CONCEPTS INFLUENCE YOUR WAY OF WINKING 
ABOUT GOVERNMENT PROGRAMS IN TERMS OF: 



TABLE V I 1-2 



RESPONSE 


DEFENSE 

(n=15) 

% 


HUMAN 

RESOURCES 

(n=ll2) 

% 


0 it puts 


73.3 


65.2 


Costs 


60.0 


64.3 


Alternatives 


80.0 


75.0 


Objectives 


73.3 


64.3 


Other 


6.7 


4.5 


None 


6.7 


10.7 



INTERNATIONAL 

(n=34) 

X 


MANAGEMENT 

(n=166) 

7, 


NATURAL 

RESOURCES 

(n=l28) 

% 


SCIENCE 4 

TECHNOLOGY 

(n=50) 

X 


50.0 


63.0 


71.9 


60.0 


50.0 


58. 7 


67.2 


50.0 


82.4 


81.3 


80.5 


80.0 


58.8 


68.7 


69.5 


62.0 


11.8 


6.0 


5.5 


12.0 


8.8 


7.8 


4.7 


10.0 



Here significantly more individuals in Management programs 
reported their thinking about outputs had been influenced than did those 
in Science and Technology programs. 
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?/5. PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THK 

QUANT 1 1’ AT IVE AND S /STEMS ANALYSIS TECHNIQUES HAVE BEEN 
HELPFUL TO YOU. 

TABLE VI I- 3 



RESPONSE 


DEFENSE 

(n=l5) 

% 


HUMAN 
RESOURCES 
(n=l 12) 

% 


INTERNATIONAL 

(n=34) 

7c 


MANAGEMENT 

(n=l66) 

7c 


NATURAL SCIENCE & 
RESOURCES TECHNOLOGY 
(n=l28) (n=50) 

% % 


Problem Structuring 


* 

20.0 


29.5 


23.5 


25.3 


34.4 


42.0 


Problem Analysis 


46.7 


48.2 


29.4 


49.4 


53.9 


66.0 


Problem Solving 


40.0 


30.4 


26.5 


37.3 


21.1 


16.0 


Communicating with Others 


60.0 


42.9 


38.2 


42.2 


46.9 


52.0 


Identifying and 
Considering Alternatives 


53.3 


59.8 


52.9 


58.4 


58.6 


70.0 


Other 


6.7 


4.5 


5.9 


3.0 


3.1 


9.0 



The observed differences are not statistically significant. This 
is due to the relatively small number of persons in some of the groups 
that appear to be different. 



* 

Frequency too small for test. 
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#8. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A DIRECT 
RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 

TABLE VI I- A 



RESPONSE 


DEFENSE 

(n=15) 

% 


HUMAN 

RESOURCES 

(n=H2) 

7c 


INTERNATIONAL 

<n=34) 

7c 


MANAGEMENT 

(n=166) 

7c 


NATURAL 

RESOURCES 

(n=128) 

7* 


SCIENCE & 

TECHNOLOGY 

(n=50) 

7 0 


Yes 


33.3 


31.2 


26.5 


25.3 


26.6 


46.0 


No 


66.7 


68.7 


73.3 


74.1 


73.4 


54.0 


No Response 


- 


.1 


- 


.6 


- 


- 



The observed differences are not statistically significant at a 
95 percent level of confidence. 
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#10. FOLLOWING YOUR EXPERIENCE WITH THE COURSE, HAVE YOU RECOMMENDED 
IT FOR OTHERS IN YOUR AGENCY? 









TABLE VI I- 5 








RESPONSE 


DEFENSE 

(n=15) 

X 


HUMAN 
RESOURCES 
(n=l 12) 

X 


INTERNATIONAL 

(n=34) 

7 . 


MANAGEMENT 

(n=166) 

7 « 


NATURAL 

RESOURCES 

(n=128) 

% 


SCIENCE & 

TECHNOLOGY 

(n=50) 

X 


Yes 


93.3 


76.8 


73.5 


68.7 


79.7 


92.0 


No 


6.7 


23.2 


26.5 


30.7 


20.3 


8.0 


No Response 


• 


- 




.6 







There is a significant difference between the responses by former 
participants in Defense programs and other programs. The size of the 
sample from Defense (15), however, suggests caution in drawing conclusions. 
The response of the Management category, however, was also significantly 
less than the Natural Resources and Science and Technology categories and 
the volume of returns was sufficient to be conclusive. 

The number of respondents in the Management program category who 
had recommended the course was also significantly lower than the average 
of all the other categories. This contradicts the pattern of uniform 
positive responses by persons in the Management program category to the 
other questions. 
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VIII. OCCUPATIONAL CATEGORIES 

Four occupational categories were established and the 
questionnaires were grouped accordingly to analyse possible 
differences in reaction to the course and its carryover. The 
categories are Financial, which includes budgetary functions, 
Line Management ; Technical, which includes elements of both 
line and staff but has the primary identifying characteristic 
of involving professional education and standards that are less 
clearly related to public and private sectors, such as medicine 
and the physical sciences; and Staff Support, which includes 
all staff functions except those indicated above. 

The questions selected for analysm and the responses are 
shown in the following tables. 
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#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF 
GREATEST PROFESSIONAL BENEFIT TO YOU? 

TABLE VIII- 1 

LINE STAFF 



RESPONSE 


FINANCIAL 

(n=l23) 

% 


MANAGEMENT 

(n=101) 

% 


TECHNICAL 

(n=50) 

% 


SUPPORT 

(n=228) 

% 


Economic Concepts 


47.2 


55.4 


52.0 


49.1 


Quantitative Techniques 


35.0 


30.7 


32.0 


28.9 


Analytic Techniques 


49.6 


57.4 


42.0 


59.2 


Information System Concepts 


37.4 


35.6 


22.0 


26.3 


Program Structuring 


47.2 


41.6 


46.0 


42.5 


PM, PFP Preparation 


19.5 


13.9 


32.0 


25.0 


Other 


10.6 


5.9 


14.0 


7.0 



Those in Staff Support occupations got significantly 
greater professional benefit from Analytic Techniques than 
did those in Technical occupations. But the Technical category 
reported significantly greater benefit from the course elements 
concerning program memoranda preparation than did those in line 
management. This preference may be related to the degree of 
specialization involved in each occupation. 



O 
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#3. 


AT THE TIME YOU LEFT THE COURSE, TO WHAT 
UNDERSTAND HOW THE CONCEPTS AND ELEMENTS 
#. 2 FIT TOGETHER TO FORM THE PPB SYSTEM? 


EXTENT DID YOU 
LISTED IN QUESTION 






TABLE 


VIII-2 






RESPONSE 




FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

7. 


TECHNICAL 

(n=50) 

% 


STAFF 

SUPPORT 

(n=228) 

% 


Good Understanding 


61.0 


45.5 


46.0 


53.9 


Fair Understanding 


39.0 


49.5 


48.0 


43.0 


Didn't Really Understand 


- 


5.0 


6.0 


2.6 




No significant difference 


, but those in 


Financial 


occupations 


seemed to be more comfortable with the material. 




#4. 


DID THE ECONOMIC CONCEPTS 
ABOUT GOVERNMENT PROGRAMS 


INFLUENCE YOUR 
IN TERMS OF: 


WAY OF THINKING 






TABLE VIII-3 






RESPONSE 




FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

% 


TECHN ICAL 
(n=50) 

% 


STAFF 

SUPPORT 

(n=228) 

7. 


Outputs 




67.5 


63.4 


66.0 


65.4 


Costs 




63.4 


61.4 


62.0 


69.3 


Alternatives 




76.4 


82.2 


80.0 


81.1 


Objectives 




64.2 


66.3 


64.0 


68.9 


Other 




3.3 


5.0 


4.0 


3.5 


None 




6.5 


5.0 


8.0 


8.3 



No significant difference. 
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#5. PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THE 

QUANTITATIVE AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN 
HELPFUL TO YOU. 



TABLE VI 1 1- 4 



RESPONSE 


FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

% 


TECHN ICAL 
(n=50) 

% 


Problem Structuring 


23.6 


30.7 


28.0 


Problem Analysis 


38.2 


47.5 


54.0 


Problem Solving 


23.6 


35.6 


22.0 


Communicating with Others 


54.5 


40.6 


44.0 


Identifying and Considering 
Alternatives 


45.5 


71.3 


66.0 


Other 


3.3 


6.9 


16.0 



The rate of selection of Problem Analysis by persons in 
Financial occupations was significantly less frequent than by 
the Staff Support group. Finance people responded significantly 
less frequently about Identifying and Considering Alternatives 
than any of the other categories. The same distinction exists 
between Financial and all other occupational areas concerning 
Identifying and Considering Alternatives. 



.. 83 

ERIC 



STAFF 

SUPPORT 

<n=228) 

% 

33.3 

58.8 

32.9 

41.7 

62.7 
3.1 
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#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT THAT 
YOU HAVE CONTINUED TO STUDY IN RELATED SUBJECTS THROUGH 
OUTSIDE COURSES, PERSONAL READINGS, ETC.? 



TABLE VIII-5 



RESPONSE 


FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

% 


TECHN ICAL 
(n=50) 

% 


STAFF 

SUPPORT 

Cn=228) 

% 


Yes 


56.9 


55.4 


62.0 


64.0 


No 


42.3 


44.6 


38.0 


34.2 


No Response 


.8 . 


- 


- 


1.8 



No significant difference. 



m. CAN YOU CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH ARE A 
DIRECT RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 

TABLE VIII-6 



RESPONSE 


FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

% 


TECHNICAL 

(n=50) 

% 


STAFF 

SUPPORT 

(n=228) 

% 


Yes 


24.4 


35.6 


28.0 


35.5 


No 


74.8 


64.4 


72.0 


64.0 


No Response 


.6 


- 


- 


.5 



No significant difference. 
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#10. FOLLOWING YOUR EXPERIENCE WITH THE COURSE, HAVE fOU RECOMMENDED IT 
FOR OTHERS IN YOUR AGENCY? 



TABLE VI I 1-7 



RESPONSE 


FINANCIAL 

(n=123) 

% 


LINE 

MANAGEMENT 

(n=101) 

% 


TECHNICAL 

(n=50) 

% 


STAFF 

SUPPORT 

(n=228) 

% 


Yes 


75.6 


74.3 


74.0 


79.4 


No 


22.8 


22.8 


24 0 


20.2 


No Response 


1.6 


2.9 


2.0 


.2 



No significant difference. 
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IX. WORKING DIRECTLY IN PPB AND NOT WORKING DIRECTLY IN PPB 



To investigate the nature and degree of difference of types 
of training carryover observed by those former participants whose 
primary duties are directly related to PPB operations, and by 
those who are not working directly In some phase of PPB, we 
grouped the questionnaires according to the answer to Question #1. 
Answers to the questions that were analyzed are shown in the tables 
below. In this series of tables a statistically significant 
difference is present when there are 10 or more percentage points 
variation in the response. 



#2. WHAT ELEMENT OR ELEMENTS OF THE COURSE HAVE BEEN OF GREATEST 
PROFESSIONAL BENEFIT TO YOU? 

TABLE IX- 1 



RESPONSE 


WORKING DIRECTLY 
IN P?3* 
(n=312) 

% 


NOT WORKING 
DIRECTLY IN PPB* 
(n=186) 

% 


Economic Concepts 


53.8 


43.5 


Quantitative Techniques 


34.9 


25.3 


Analytic Techniques 


55.8 


49.5 


Information System Concepts 


27.6 


34.9 


Program Structuring 


45.2 


39.8 


PM, PFP Preparation 


25.3 


17.7 


Other 


7.7 


7.0 



Only Economic Concepts were significantly greater benefit to 



those working in PPB. 

* 

The sum of '’n' 1 does not equal 503 due to no response or unusable 
response to Question #1 by some respondents. 

Responses significantly different. 

o 
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# 5. PLEASE INDICATE SITUATIONS WHERE APPLICATION OF THE 

QUANT rr AT £VE AND SYSTEMS ANALYSIS TECHNIQUES HAVE BEEN 
HELPFUL TO YOU. 



TABLE IX- 2 



RESPONSE 


WORKING DIRECTLY 
IN PPB 
(n=312) 

% 


NOT WORKING 
DIRECTLY IN PPB’ 
(n=186) 

7o 


Problem Structuring 


35.3 


18.3 


Problem Analysis 


55.4 


43.5 


Problem Solving 


27.9 


31.2 


Comr.nicating with Others 


45.8 


43.0 


Identifying and Considering 
Alternatives 


59.3 


61.8 


Other 


4.2 


7.5 



The techniques were significantly less helpful to those not 
in PPB in situations involving Problem Structuring and Problem 
Analysis. 

** 

Responses significantly different. 



o 
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#6. IF YOU WERE GOING TO ATTEND THE COURSE NOW, IS THERE ANY 
PORTION WHERE YOU FEEL THE EMPHASIS SHOULD BE CHANGED? 

TABLE IX- 3 

(Totals do not equal 10D percent since approximately 10 
> percent did not answer each portion.) 



RESPONSE 


WORKING DIRECTLY 
IN PPB 
(n-312) 

% 


NOT WORKING 
DIRECTLY IN PPB 
(n«186) 

% 


ECONOMICS 


Increased 


19.2 


17.7 


Decreased 


28.8 


25.3 


No Change 


43.3 


45.7 



MATHEM AT ICS/ STAT 1ST ICS 
Increased 
Decreased 
No Change 
CASE STUDIES 
Increased 
Decreased 
No Change 
SYSTEMS ANALYSIS 
Increased 
Decreased 
No Change 
INFORMATION THEORY 



Increased 


31.1 


29.0 


Decreased 


16.3 


10.8 


No Change 


39.7 


45.2 



O 

ERIC 



19.6 
37.8 

33.7 



22.0 

33.3 

32.8 



38.8 

15.4 

37.2 



45.7 

16.1 

29.0 



50.3 

8.0 

29.5 



48.9 

4.3 

35.5 
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RESPONSE 

PROGRAM STRUCTURING 
Increased 
Decreased 
No Change 

PREPARATION OF PM, PFP 
Increased 
Decreased 
No Change 

OTHER 

Increased 
Decreased 
No Change 



WORKING DIRECTLY 
IN PPB 
(n=312) 

% 

47.4 

11.2 

32.4 

50 5 
9.3 

28.5 

8.7 

1.0 

.64 



NOT WORKING 
DIRECTLY IN PPB 
(n-186) 

% 

45.7 

9.7 
33.3 

43.0 
7.5 

36.0 

3.8 
.5 



v 



There were strong suggestions for change in some subject 
matter categories, and both groups registered essentially similar 
opinions. 
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#7. DID THE COURSE STIMULATE YOUR INTEREST TO THE EXTENT 
THAT YOU HAVE CONTINUED TO STUDY IN RELATED SUBJECTS 



THROUGH 


OUTSIDE COURSES, PERSONAL READINGS, ETC.? 
TABLE IX- 4 




RESPONSE 


WORKING DIRECTLY 


NOT WORKING 


IN PPB 


DIRECTLY IN PPB 




(n=312) 


(n=186) 




% 


% 


** 

Yes 


67.0 


47.8 


No 


31.7 


50.5 


No Response 


1.3 


1.7 


The group not in PPB pursued further study in this 


area with 


significantly less frequency. 




#8. CAN YOU 


CITE ANY SPECIFIC EXAMPLES OF OUTPUT WHICH 


ARE A 


DIRECT 


RESULT OF OR LARGELY INFLUENCED BY THE COURSE? 
TABLE IX- 5 


RESPONSE 


WORKING DIRECTLY 


NOT WORKING 


IN PPB 


DIRECTLY IN PPB 




(n-312) 


(n=l86) 




% 


7o 


** 

Yes 


39.4 


26.9 


No 


40.1 


51.6 


No Response 


20.5 


21.5 



Although the frequency of output related to the course by 
those not In PPB has been significantly lower, the number of persons 
not in PPB who reported such output indicates a substantial amount 
of carryover to other functions. 




** 



Response significantly different, 
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#10. FOLLOWING YOUR EXPERIENCE WITH THE COURSE, HAVE YOU 
RECOMMENDED IT FOR OTHERS IN YOUR AGENCY? 



TABLE IX- 6 



RESPONSE 


WORKING DIRECTLY 


NOT WORKING 




IN PPB 


DIRECTLY IN PPB 




(n=312) 


(n=186) 


** 


% 


% 


Yes 


80.1 


69.9 


No 


17.9 


28.0 


No Response 


2.0 


2.1 



While those in PPB recommended the course to others with 
significantly higher frequency, the volume of endorsement by those 
less directly concerned is impressive. 



** 

Responses significantly different. 
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APPENDIX B 
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PPB Seminar 

Follow-up Questionnaire 



CODE (leave blank) 



Name : _ 

Employing Organization: (Dept, or Independent Agency) 



Ma}or Organization Sub-division: 

Business Address: Office Phone Number: 

Zip Code : 

Present Position Title: 

Present Grade: 

Position Title at Time of Course: 

Grade at Time of Course: 

Age: Years of Government Service: Civilian: Military :_ 

Please indicate the occupational categories in which you have had 
experience (Government and other) and the number of years in each 
category. 



lJ 


Financial or Budget Operations 


years 


LJ 


Personnel Operations 


years 


i 7 


Management Analysis 


years 


LJ 


Data Processing Operations 


years 


n 


Line Management 


years 


i 7 


Other (Please Specify) 


years 



ERIC 
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Education : 



Credit 





Please list other Government sponsored courses related to PPBS you have 
attended in the past two (2) years. 



1. Are you currently working directly in some phase of PPB? 

/ 7 Yes / 7 No 

If yes, answer la & lbj If no, answer 1c. 

la. Is your work principally in 

Planning _/~/ Programming / / Budgeting / / ? 

lb. Is it generally Managerial / / Analytical / / ? 

lc. Is your work indirectly supportive of PPB functions 
(PM, PFP, Spec. Study Input, Data Collection, etc.?) 

iZJ Yes / 7 No 

2. What element or elements of the course have been of greatest professional 
benefit to you? (Check as many as apply) 

/ / Economic Concepts 

/ / Quantitative Techniques 

[_ / Analytic Techniques 

(List continued) 
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/ / Information System Concepts 

!_ / Program Structuring 

/ / PM, PFP Preparation 

f_ 7 Other 

At the time you left the course, to what extent did you understand 
how the concepts and elements listed in Question #2 fit together to 
form the PPB system? (Check appropriate block) 

/ / Good Understanding 

/, / Fair Understanding 

/ 7 Didn’t Really Understand 

Did the economic concepts influence your wey of thinking about 
government programs in terms of: (Check as many as apply) 



Yes_ 


No 




£_ 


j 


U 


Outputs 


r_ 


7 


rj 


Costs 


r 


7 


rj 


Alternatives 


r_ 


7 


rj 


Objectives 


r 


7 


o 


Other 



Please indicate situations where application of the quantitative and 
systems analysis techniques have been helpful to you. (Check as many 
as apply) 

£ ^ / Problem Structuring 

/ / Problem Analysis 

/ / Problem Solving 

/ / Communicating with Others 

J_ / Identifying and Considering Alternatives 

/ / Other 
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jf you were going to attned the course now ) is tbero sny portion 
where you feel the emphasis should be changed? (Check appropriate 
column) 



No 

Increased Decreased Change 



Economics 

Mathematics /Statistics 
Case Studies 
Systems Analysis 
Information Theory 
Program Structuring 
Preparation of PM, PFP 
Other: 



7. Did the course stimulate your interest to the extent that you have 
continued to study in related subjects through outside courses, 
personal readings, etc.? 

/ 7 Yes / 7 No 



Subject ( s) 



8. Can you cite any specific examples of output which are a direct 
result of or largely influenced by the course? (Brief comment or 
attach sample) 



9. Are there any particular facets of PPB for which you think follow-up 
training courses should be developed that would be useful to you? 



10. Following your experience with the course, have you recommended it for 

others in your agency? 

/ / Yes / / No 

It. With the benefit of time and the opportunity to digest and use the 

training, do you have other observations or suggestions regarding the 

PPB Seminar? 



THANK YOU FOR COMPLETING AND RETURNING THIS QUESTIONNAIRE. 

1Q0 



> 



AP»?£»OT.< 0 



*v 




* 
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GUEST FACULTY 



Throughout the history of the PPBS seminar there has been 
a sustained effort to obtain the best available experts to serve 
as guest faculty. In addition to providing guidance about the 
structure and content of the seminar, key Bureau of the Budget 
staff members have played an active role as guest lecturers. Also, 
knowledgeable officials from throughout Government have partici- 
pated. Noted authors and teachers have been brought in from 
university campuses and advice and assistance has bean obtained 
from experienced senior analysts with consulting firms. 

The following individuals have served as guest faculty for 
the PPB seminar, some on many occasions: 



FEDERAL GOVERNMENT REPRESENTATIVES 



Jack Kratchman 

Chief, Program Plans Branch 

Div. of Plans St Reports 

U, S. Atomic Energy Commission 



William A. Carlson 
Planning, Evaluation 5* 



Program Staff 
Department of Agriculture 



Milton Searle 

Chief , Economics Branch 

Div. of Operation Analysis & 



Howard W. Hjort 

Director, Planning, Evaluation & 



Forecast 

Atomic Energy Commission 



Programming Staff 
Department of Agriculture 



Jerome Snyder 

Plans Analyst 

Div. of Plans & Reports 

U. S. Atomic Energy Commission 



Steven Dola 

Systems Analysis Group 
Department of the Army 
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Lt. Col. Marvin S. Weinstein 
Office, Assistant Secretary of 
the Army (FM) 

James DeLong 

Member, Program Evaluation 
Staff 

Bureau of the Budget 

James Duesenberry (Dr.) 

Member, Council of Economic 
Advisors 

Paul Feldman (Dr.) 

Program Evaluation Staff 
Office of the Director 
Bureau of the Budget 

C. William Fisher '* * 
Assistant Chief 
International Division 
Bureau of the Budget 

Robert Gallamore (Dr.) 

Program Evaluation & 

Review Staff 
Bureau of the Budget 

John Haldi (Dr.) 

Chief 

Program Evaluation Staff 
Bureau of the Budget 

Fred S. Hoffman 
Assistant Director 
Bureau of the Budget 

Frank W. Krause 
Director 

Resources System Staff 
Office of Budget Review 
Bureau of the Budget 

Roy W. Niemela (Dr. ) 

Program Evaluation Staff 
Bureau of the Budget 

David A. Page 

Program Evaluation Staff 

Bureau of the Budget 



ERIC 



John Roose 
Budget Examiner 
Natural Resources Programs 
Division 

Bureau of the Budget 

Nestor E. Terleckyj 
Program Evaluation Staff 
Bureau of the Budget 

Ralph R. Thelwell 
Management Analyst 
Bureau of the Budget 

John Wirt 

Program Evaluation Staff 
Bureau of the Budget 

Charles J. Sparks 

Deputy Director 

Bureau of Management Services 

Civil Service Commission 

George Green (Dr. ) 

Office of Business Economics 
Department of Commerce 

Laurence E. Lynn 

Head, Office of Transportation & 
Strategic Mobility 

Office of Deputy Assistant Secretary 
for Economic Development 
Department of Defense 

Dwight Greene 
Director 

Management Information Center 
Environmental Science Service 
Administration 

Malcolm Kirby (Dr.) 

Management Sciences Staff 
U. S. Forest Service 

Daniel J. Johnson 

Office of Programs & Policy Planning 
General Services Administration 

Edmond J. Rouhana 

Director of Programs & Policy Planning 
General Services Administration 
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Robert N. Grosse (Dr.) 

Deputy Assistant Secretary 
for Program Coordination 
Department of Health, Education & 
Welfare 

Gerald Sparer 
Director of Program Review 
Department of Health, Education & 
Welfare 

Dwight Rettie 
Director, Division of Land 
Development 

Department of Housing and Urban 
Development 

Robert R. Lee (Dr.) 

Director 

Idaho Water Resources Board 

John A. Carver, Jr. 

Under Secretary of the Interior 
Department of the Interior 

John Dawson 

Office of Program Analysis 
Department of the Interior 

J. Karl Lee 

Office of the Assistant 
Secretary 

Water Power Development 
Department of the Interior 

Harley Mack 
Assistant Director 
Office of Program Analysis 
Department of the Interior 

Robert A. Levine 

Chief, Research & Plans Division 
Office of Research, Plans, Programs & 
Evaluation 

Office of Economic Opportunity 

Robert N . Anthony 

Assistant Secretary of Defense 



Emerson Markham 

Director, Programming Division 
U. S. Post Office Department 

Benjamin Mandel 

Director, Office of Statistical 
Programs 

U. S. Post Office Department 

Robert Miller 
Chief 

Planning & Programming Staff 
U. S. Information Agency 

Joseph T. McDonnell 
Congressional Staff 
Senate Appropriations Committee 

William H. Boswell (Capt.) 

U. S. C. G. 

Office of Planning & Program 
Review 

Department of Transportation 

Vorley M. Rexroad 
Appropriations Committee 
United States Senate 

Samuel M. Greenhouse 
Administrator's Advisory Council 
Veterans Administration 

Willis Underwood 
Chairman 

Administrator's Advisory Council 
Veterans Admin istat ion 

Richard P. Nalesnik 
Director 

Program Review a Analysis 
Federal Water Pollution Control 
Admin is tat ion 

Robert Chart rand 
Information Science Specialist 
Library of Congress 




William Lord (Dr.) 

Staff Economist 

Office of Secretary of the Army 
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